LLMs need vastly more data than children to learn language. Understanding why could help us create more efficient models--and reveal more about developing minds. People have been talking to each other for at least 100,000 years, as best we can tell. And in all that time, there has been only one thing in the world that could learn a human language to perfect fluency: a human child. Four short years after the release of ChatGPT, many of us now take it for granted that we can converse naturally with our phones or computers. LLMs like Claude, DeepSeek, and OpenAI's GPT models are fluent and flexible enough to masquerade convincingly as humans. But peek behind the computational curtain, and there's a catch: Teaching a computer to use human language still requires an inhuman amount of data. An LLM can easily churn through a hundred thousand times more words than a person will experience in the process of mastering their mother tongue--and way more than children might hear by their first birthday, when they typically start to grab hold of language. "The progress recently has been amazing," Michael C. Frank, a cognitive scientist at Stanford University, says of LLMs. "But we still have to burn down a forest and scrape the entire sum of all human knowledge to re-create this milestone that happens in our living rooms over the course of a year."

For this experiment, the researchers relied on 61 hours of video from a helmet camera worn by a child who lives near Adelaide, Australia. That child, Sam, wore the camera off and on for one and a half years, from the time he was six months old until a little after his second birthday. The camera captured the things Sam looked at and paid attention to during about 1% of his waking hours. It recorded Sam's two cats, his parents, his crib and toys, his house, his meals, and much more. "This data set was totally unique," Lake says.

The explosive growth in artificial intelligence in recent years -- crowned with the meteoric rise of generative AI chatbots like ChatGPT -- has seen the technology take on many tasks that, formerly, only human minds could handle. But despite their increasingly capable linguistic computations, these machine learning systems remain surprisingly inept at making the sorts of cognitive leaps and logical deductions that even the average teenager can consistently get right. In this week's Hitting the Books excerpt, A Brief History of Intelligence: Evolution, AI, and the Five Breakthroughs That Made Our Brains, AI entrepreneur Max Bennett explores the quizzical gap in computer competency by exploring the development of the organic machine AIs are modeled after: the human brain. Focusing on the five evolutionary "breakthroughs," amidst myriad genetic dead ends and unsuccessful offshoots, that led our species to our modern minds, Bennett also shows that the same advancements that took humanity eons to evolve can be adapted to help guide development of the AI technologies of tomorrow. In the excerpt below, we take a look at how generative AI systems like GPT-3 are built to mimic the predictive functions of the neocortex, but still can't quite get a grasp on the vagaries of human speech.

How can machines understand language? is a question that many have asked, and represents an important facet of artificial intelligence. Large language models like ChatGPT seem to understand language, but as has been pointed out (Bender and Koller, 2020; Bisk et al., 2020), even large, powerful language models trained on huge amounts of data are likely missing key information to allow them to reach the depth of understanding that humans have. What information are they missing, and, perhaps more importantly, what information do they have that enables them to understand, to the degree that they do? Current computational models of semantic meaning can be broken down into three paradigms: distributional paradigms where meaning is derived from how words are used in text (i.e., the notion that the meaning of a word depends on the "company it keeps," following Firth (1957)) meaningfulness of language lies in the fact that it is about the world (Dahlgren, 1976) and grounded paradigms are where aspects of the physical world are linked to language (i.e., the symbol grounding problem following Harnad (1990)) formal paradigms where meaning is a logical form (e.g., first order logic as in L.T.F.

In contrast, one of the specific language-learning chatbots is LangAI, launched in March by Federico Ruiz Cassarino. Mr Ruiz Cassarino drew on his own experiences of learning English after moving from Uruguay to the UK. His English skills improved dramatically from speaking every day, compared to more academic methods. He's now using his own app to work on his Italian.

Large language models (LLMs), like OpenAI ChatGPT and Google LaMDA, are impressive, being competent in many aspects. At the same time, LLMs are incompetent in many ways. LLMs are evolving, and new players are joining. What further progress may be possible? Moreover, we may ask a question relevant to almost all players in the world of LLMs, from students, researchers, engineers, entrepreneurs, venture capitalists, officers, to the public crowd: Where is the boundary for large language models?

Unlike the carefully scripted dialogue found in most books and movies, the language of everyday interaction tends to be messy and incomplete, full of false starts, interruptions and people talking over each other. From casual conversations between friends, to bickering between siblings, to formal discussions in a boardroom, authentic conversation is chaotic. It seems miraculous that anyone can learn language at all given the haphazard nature of the linguistic experience. For this reason, many language scientists -- including Noam Chomsky, a founder of modern linguistics -- believe that language learners require a kind of glue to rein in the unruly nature of everyday language. And that glue is grammar: a system of rules for generating grammatical sentences.

Unlike the carefully scripted dialogue found in most books and movies, the language of everyday interaction tends to be messy and incomplete, full of false starts, interruptions and people talking over each other. From casual conversations between friends, to bickering between siblings, to formal discussions in a boardroom, authentic conversation is chaotic. It seems miraculous that anyone can learn language at all given the haphazard nature of the linguistic experience. For this reason, many language scientists – including Noam Chomsky, a founder of modern linguistics – believe that language learners require a kind of glue to rein in the unruly nature of everyday language. And that glue is grammar: a system of rules for generating grammatical sentences.

Unlike the carefully scripted dialogue found in most books and movies, the language of everyday interaction tends to be messy and incomplete, full of false starts, interruptions, and people talking over each other. From casual conversations between friends, to bickering between siblings, to formal discussions in a boardroom, authentic conversation is chaotic. It seems miraculous that anyone can learn language at all given the haphazard nature of the linguistic experience. For this reason, many language scientists--including Noam Chomsky, a founder of modern linguistics--believe that language learners require a kind of glue to rein in the unruly nature of everyday language. And that glue is grammar: a system of rules for generating grammatical sentences.

As technology evolves, the ways people interact with it also changes. Internet searches have become exponentially easier. It wasn't long ago that what you typed into a search bar had to be very specific and would often yield strange and unrelated results. Today, with more advanced predictive text features, it seems search engines (or anywhere you can type text or leverage natural language) can almost read your mind and know exactly what you are looking for. Language is a vital part of human connection.